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Abstract 

A common goal in an experimental physics analysis is to extract information from a reaction with multi-dimensional 
kinematics. The preferred method for such a task is typically the unbinned maximum likelihood method. In fits 
using this method, the likelihood is a goodness-of-fit quantity in that it effectively discriminates between available 
hypotheses; however, it does not provide any information as to how well the best hypothesis describes the data. In 
this paper, we present an ad-hoc procedure for obtaining % 2 /n.d.f. values from unbinned maximum likelihood fits. 
This method does not require binning the data, making it very applicable to multi-dimensional problems. 
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1. Introduction 

In many physics analyses, one seeks to extract 
information from a reaction with multi-dimensional 
kinematics. Fits to binned data are often limited by 
statistical uncertainties; thus, use of the unbinned 
maximum likelihood method is generally preferred. 
This method is well suited to obtaining estimators 
for unknown parameters in a hypothesis' probability 
density function (p.d.f.). It is also an excellent way 
of discriminating between hypotheses; however, it 
does not provide a means of determining how well 
the best hypothesis describes the data. 

In this paper, we describe an ad-hoc procedure 
for obtaining a x 2 /n.d.f. goodness-of-fit measure- 
ment of a hypothesis whose parameters were deter- 
mined using an unbinned maximum likelihood fit 
without having to bin the data. This quantity is 
not meant to replace the likelihood. The likelihood 
should still be used to determine which hypothesis 
best describes the data and to obtain estimators for 
any unknown parameters, i.e. extract any physical 



observables. The goodness-of-fit quantity obtained 
using our method would simply be used to judge 
how well the best hypothesis describes the data. 

In our approach, event-by-event standardized 
residuals are used to obtain the global x 2 /n.d.f .. 
These residuals can also be used for diagnostic pur- 
poses to study the goodness-of-fit as a function of 
kinematics or detector components. Determining 
where in phase space a fit fails to describe the data 
is vital to understanding what physics has not been 
accounted for in the p.d.f. Identifying regions or 
components of the detector where a fit fails to de- 
scribe the data could help diagnose unaccounted for 
inefficiencies in the acceptance calculation. 

Other authors have also developed methods for 
obtaining goodness-of-fit quantities from unbinned 
data. The methods closest to ours are based on the 
nearest neighbors to a given event |1I2I3| . However, 
there are several other recent articles on other proce- 
dures as well 4 5 6. While these and other authors 
have developed methods for obtaining goodness-of- 
fit quantities from unbinned maximum likelihood 
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fits, the advantage of our approach lies in the event- 
by-event residuals. It is also straight-forward to ap- 
ply our method to most experimental physics anal- 
yses. 

2. The Method 

Consider a data set composed of n total events, 
each of which is described by m coordinates, £. The 
coordinates can be masses, angles, energies, etc. Two 
separate hypotheses have been proposed to describe 
the data. Their probability density functions will be 
denoted by 



(1) 



where fh are the functional dependencies on the co- 
ordinates, £, and (possible) unknown parameters, 
dh, of hypothesis h (h = 1,2). 

Using these p.d.f.'s, unbinncd maximum likeli- 
hood fits can easily be performed to obtain estima- 
tors, dh, for the unknown parameters, dh, of each 
hypothesis. The likelihoods from these fits, Ch, can 
be used to determine which hypothesis provides the 
better description of the data; however, the following 
simple question about this hypothesis can not yet 
be answered: "How well does it describe the data?" 

The aim of this procedure is to obtain a \ 2 /n.d.f. 
goodness-of-fit measurement of a hypothesis whose 
parameters were determined using an unbinned 
maximum likelihood fit. We first need to define a 
metric for the space spanned by the "relevant" co- 
ordinates of £, i.e. all coordinates on which the p.d.f 
has functional dependence. A reasonable choice is 
to use S^i/H 2 ., where IZk is the maximum possible 
difference between the coordinates, of any two 
events in the sample. Using this metric, the distance 
between any two events, dij, is given by 



E 



(2) 



where the sum is over all relevant coordinates (dis- 
cussed above). 

For each event, we then compute the distance, rj, 
to the n c 'th closest event using pi). The value of n c 
is discussed below. Thus, a hypersphere centered at 
the i th event with radius r, contains n c data events 
(excluding the event itself) . In this way, we can cal- 
culate the density of data events at the point & . By 



comparing these density calculations to those pre- 
dicted by a given hypothesis, we can determine how 
well the hypothesis describes the data without re- 
sorting to binning. 

For each event, the standardized residual, z, for 
any hypothesis is then calculated according to 



(3) 



where n mi (n Pi ) is the number of measured (pre- 
dicted) events contained within a hypersphere of ra- 
dius T{ centered at £j and a mi (a Pi ) is the uncertainty 
on n m .(n p .). 

For the case where the data set contains no 
background events (dealing with backgrounds is 
discussed in Section ', 



3.3 1, n m , = n c and cr mi = 



provided n c is chosen to be large enough for the 
Gaussian approximation to hold. The number of 
predicted events, n Pil is calculated for hypothesis h 
as 



J e fh(!;,dh)d£ 
J fh(€,&h)d£ 



(4) 



where the integral in the numerator is over the hy- 
persphere of radius centered at £j discussed above. 

In principle, there are cases where the integrals 
in Q can be performed analytically; however, once 
detector acceptance is included, for which an ana- 
lytic expression typically is not known, they must 
be done using the Monte Carlo technique. For these 
cases, n p can be calculated by generating n mc Monte 
Carlo events according to £ phase space which leads 
to the following approximation of Q : 



E fh^,j,d h )Q{r i -r j ) 

_i_ 

k 



(5) 



where the Heaviside function, Q(ri — rj), is used to 
restrict the sum in the numerator to Monte Carlo 
events within the hypersphere. The statistical uncer- 
tainty in the Monte Carlo calculation of n p would be 
a p = n p l 'y/n'mc, where n r mc is the number of Monte 
Carlo events within the hypersphere. We note here 
that if n r mc is small, then the uncertainty is better 
approximated by 1 + \Jn r mc + 0.75 [7] . 

If the hypothesis does in fact provide a good de- 
scription of the data, then the values of z 2 obtained 
from ^ will follow a \ 2 distribution with one de- 
gree of freedom. The overall goodness-of-fit can be 
obtained as follows: 



2 



n 



X z /n.d.f. = — ^zf, (6) 



We chose to use the values p® = 0.65, p°_i = 0.05 
and Rep^ — 0.10 for our simulated data set. 



where n par is the number of free parameters present 
in the hypothesis' p.d.f. Hypotheses which describe 
the data well will yield values of \ 2 / n -d-f- ~ 1, while 
those which do not describe the data well will yield 
larger values. 



3. Example Application 

As an example, we will consider the reaction 
7P — > pui in a single (s, t) bin, i.e. a single center-of- 
mass energy and production angle bin (extending 
the example to avoid binning in production angle, 
or t, is discussed below). The u> decays to 7r + 7r~7r° 
about 90% of the time; thus, we will assume we have 
a detector which has reconstructed jp — > p7r + 7r~7r° 
events. Below we will analyze a simulated u> photo- 
production data set. The goal of our model analysis 
is to extract the to polarization observables known 
as the spin density matrix elements, denoted by 
P°MM' (discussed below) . 

In terms of the mass of the 7r + 7r~7r° system, 7713^, 
the u> events were generated according to 3-body 
phase space weighted by a Voigtian (a convolution of 
a Breit-Wigner and a Gaussian) to account for both 
the natural width of the u> and detector resolution. 
For this example, we chose to use a = 5 MeV/c 2 
for the detector resolution. The goal of our analysis 
is to extract the three measurable elements of the 
spin density matrix (for the case where neither the 
beam nor target are polarized) traditionally chosen 
to be Pq , p\_i and Rep\ . These can be accessed 
by examining the distribution of the decay products 
(•7r + 7r - 7r°) of the uj in its rest frame. 

For this example, we chose to work in the helicity 
system which defines the z axis as the direction of 
the lo in the overall center-of-mass frame, the y axis 
as the normal to the production plane and the x axis 
is simply given by x — y x z. The decay angles 9, (f> 
are the polar and azimuthal angles of the normal 
to the decay plane in the u> rest frame. The decay 
angular distribution of the 10 in this frame is then 
given by [S] 

W(9, $ = ~ Q(l - p° 00 ) + i(3p° - 1) cos 2 9 
- p\_ x sin 2 9 cos 20 - V2Rep° w sin 29 cos (p) . (7) 



3.1. No Background, 100% Acceptance 

We will begin by considering the simplest case 
where the signal is able to be extracted without 
any background contamination and our detector ef- 
ficiency is 100%. For this study, a simulated data set 
of 10,000 events was generated following the crite- 
ria described above (see Figure [T]). To facilitate the 
calculation of 100,000 Monte Carlo events were 
generated using the same distribution as the 
data; however, the decay distribution was generated 
according to flat phase space in cos 9 and 4>. 




c'os(6) 

(b) 

Fig. 1. (Color Online) Events generated without background 
and with 100% acceptance, (a) Mass of the 7r+7r~7r° system 
(GeV/c 2 ). (b) Decay angular distribution, 4> (radians) vs 
cos 9. 
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The likelihood function required to extract the 
spin density matrix elements is simply 



C = ]JW{9 i ,cj> i ), 



(8) 



where W is the decay angular distribution defined 
in ([T]) . To obtain estimators for the p° MM , elements, 
we minimize 



i 

which yields the values 
p° m = 0.649 ± 0.009 
pl^ = 0.043 ±0.005 
Rep° w = 0.103 ±0.004. 



(9) 

(10a) 
(10b) 
(10c) 



These values are in good agreement with the values 
used to generate the data. 

At this point in our model analysis, we have ex- 
tracted the physical observables we are interested 
in. The spin and parity of the u) meson are well es- 
tablished; however, what if we were analyzing a less 
well-known meson? In that case, we may not be cer- 
tain that J p = l~, i.e. the use of ^ might not be 
justified. Prior to publishing our results, it would be 
reassuring to know that the fit performed using Q 
does in fact provide a good description of our data. 

3.1.1. Applying the Procedure 

For this analysis, the relevant coordinates are the 
decay angles 9 and 0, since these are the only kine- 
matic quantities for which the p.d.f. defined in ([7| 
has dependence. The metric defined in ^ is then 
given by 



4 = 



^cos ( 



COS( 



2tt 



(11) 



for this analysis. 

For each simulated event, we find the distance to 
the n c 'th nearest neighbor event, r^. For now, we'll 
proceed using n c = 100, determining what to use for 
this value is discussed in detail in Section [3.1. 21 The 
phase space Monte Carlo events we generated are 
then used to obtain the predicted number densities 
according to 



_J_ 

E w(e k ,<f> k ) 

k 



(12) 



using the p° MM i values given in ( 10 1. For this exam- 
ple, the 100% acceptance makes explicitly comput- 
ing the sum in the denominator unnecessary; how- 
ever, in the examples that follow it will be required. 

The standardized residuals, z, for this hypothesis 
are obtained from |3]) as 



(100 ~n Pi f 
100 + n„. 



(13) 



where n p . is obtained from ( [12] ). Figure [2] shows 



the x 2 , pull and confidence level distributions ob- 
tained for our simulated data set. The values are 
in excellent agreement with the theoretical distribu- 
tions, i.e. they do follow a \ 2 distribution. The to- 
tal x 2 is obtained by summing over all events. For 
this example, the value is 10344; thus, x 2 /n.d.f. = 
10344/(10000 - 3) = 1.035; a clear indicator that 
this hypothesis provides an excellent description of 
the data. 

It is also instructive to examine hypotheses 
which do not perfectly describe the data. If the 
likelihood is maximized while requiring p^-i = 0> 
then the spin density matrix elements extracted 
are pg = 0.650 ± 0.009 and Rep% = 0.110 ± 0.004. 
Figure |3] shows the x 2 , pull and confidence level dis- 
tributions obtained for our simulated data set under 
this hypothesis. Since the data were generated with 
Pi-i — 0.05, this fit provides a fair description of 
the data; the total X 2 /n.d.f. = 14527/(10000-2) = 
1.453. 

Maximizing the likelihood while requiring 
= Rep% = yields pg = 0.651 ± 0.009. 
Figure |4] shows the \ 2 \ pull an( l confidence level 
distributions obtained for our simulated data 
set under this hypothesis. This fit clearly pro- 
vides a poor description of the data; the total 
X 2 /n.d.f. = 36118/(10000 - 1) = 3.612. 

The total x 2 /n.d.f . values do provide a goodness- 
of-fit value which accurately indicates how well the 
fit describes the data. Thus, these values can be used 
to ascertain the quality of the hypotheses. It is also 
interesting to note that the pull and confidence level 
distributions are also good indicators. One could 
plot these quantities vs. kinematic variables to de- 
termine where a given hypothesis fails to describe 
the data. 



3.1.2. Choosing a Value for n c 

In this section, we will examine the value of n c 
(the number of nearest neighbor events used to de- 
termine the values of the radii ri). Figure [5j a) shows 
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X 1 P ul1 distribution confidence level 



(a) (b) (c) 



Fig. 2. (Color Online) Events generated without background and with 100% acceptance. The plots represent the (a) \ 2 (b) 
pull and (c) confidence level distributions obtained from a fit to (I7j. The red lines represent the theoretical distributions, which 
contain no free parameters. See text for details. 




Fig. 3. (Color Online) Events generated without background and with 100% acceptance. The plots represent the (a) \ 2 (b) 
pull and (c) confidence level distributions obtained from a fit to ||7l in which the pj-i off-diagonal element was constrained 
to be zero. The red lines represent the theoretical distributions, wmch contain no free parameters. See text for details. 




Fig. 4. (Color Online) Events generated without background and with 100% acceptance. The plots represent the (a) \ 2 3 ) 
pull and (c) confidence level distributions obtained from a fit to |7| in which both off-diagonal elements were constrained to 
be zero. The red lines represent the theoretical distributions, which contain no free parameters. See text for details. 
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Fig. 5. (Color Online) Events generated without background and with 100% acceptance, (a) x 2 /n.d.f. vs. ^ for fits with all 
three spin density matrix parameters free, (b) x 2 / n -d-f- vs. for fits with both off-diagonal spin density matrix elements 
constrained to be zero. 



the x 2 / n -d-f value obtained for two choices of n c 
(n c = 100 and n c = 50) for various event sample 
sizes. The data sets (with sample sizes ranging from 
50 to 10,000 events) were all generated from the 
same parent distributions as in Section 3.1 Each 



data set was fit individually to extract the p%M' 
values used to obtain the x 2 's. 

Figure[5ja) clearly shows that the \ 2 /n.d.f value 
obtained does depend on the choice of n c , if n c is cho- 
sen to be greater than about 2% of the total number 
of events. For values of n c less than 2% of the event 
sample size, the x 2 /n.d.f . is quite stable. This be- 
havior is expected. If n c is large relative to n, then 
the method is averaging over large fractions of phase 
space; thus, finer structure in the physics will not 
be properly accounted for. We also note here that 
X 2 — > as n c — ► n regardless of the quality of the 
fit, due to how n p is calculated (see H). 

Figure[5jb) shows the value of x 2 /n.d.f obtained 
for the same two choices of n c for various event 
sample sizes for fits to the data which constrain 
Pi_! = Repi — 0. The values of x 2 / n -d.f. are again 
stable when n c is chosen to be less than 2% of the 
data; however, the values depend on the choice of n c . 
This behavior is again expected; a similar effect can 
be seen in fits to binned data for which the value of 
X 2 /n.d.f depends on the number of bins. The larger 
the value of n c , the larger the x 2 / n -d-f- can be. 

Consider a kinematic region where the number 
density of the data is high, leading to a small value 
of r,. For this case, the largest obtainable value for 



lim z> 

r-i->0 



lim 



(n c - n Pi )' 



+o n c + a 



(14) 



Pi 



Thus, there are two competing factors which should 
be considered when choosing the value of n c . The 
ratio of n c /n must be small enough to permit a 
true comparison of the finer structure of the physics; 
however, the value of n c must be large enough such 
that the relative statistical uncertainties do not for- 
bid larger values of z. Typically, these considerations 
will dictate n c be about l%-2% of the event sample 
size. If this results in n c being less than 50, i.e. if 
n < 2500, then this method will most likely be less 
effective. Of course, this statement is not all encom- 
passing; in practice, it will depend on the physics. 



3.1.3. A X 2 



-A log C 



It is also useful to examine how changes in the 
likelihood map onto changes in the x 2 values ob- 
tained using this method. Here we trace out both 
goodncss-of-fit quantities near the optimal value of 
Pq . Figure [6] shows the changes in — ln£ and x 2 vs 

~'(n= 10,000 
— InC docs 



3.1 



Pq for the data set used in Section 
and n c — 100). Clearly, an increase in 
corresponds to an increase in x 2 

3.2. Including Detector Acceptance 



We will now extend our example by including de- 
tector acceptance. The physics used in this section 

however 



will be identical to that of Section 3.1 



we 



will now include a detector efficiency, 77, given by 



(i 



0.65 
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Fig. 6. (Color Online) Events generated without back- 
ground and with 100% acceptance. The quantities plotted 
are — Aln£ vs. p^j (red triangles) and Ax 2 /100 vs. Pq 
(black squares). 



Fig. 7. (Color Online) <j> (radians) vs cos 9: Events generated 
without background and with detector acceptance given by 



»/(M) = -(2-|cOfl08M|). 



(15) 



We again generated 10,000 data events and 100,000 
Monte Carlo events from the same parent distribu- 
tions as in Section [3~T1 convolved with the detector 
acceptance given in (15 1. Figure [7] shows the de- 
cay angular distribution of this data set. The effects 
of the detector acceptance are clearly visible when 
compared with Figure [T] 

An unbinncd maximum likelihood fit was per- 
formed and the following values were obtained for 

p° ao = 0.634 ± 0.009 
p?_ a = 0.055 ±0.005 
Rep° w = 0.096 ± 0.004. 



(16a) 
(16b) 
(16c) 



To obtain % 2 values, we apply the procedure just as 
in Section 3.1 Figure [8]shows the x 2 > pull and confi- 
dence level distributions obtained for this simulated 
data set. The values are again in excellent agree- 
ment with the theoretical distributions. The total 
X 2 /n.d.f. = 9715.65/(10000 - 3) = 0.972, indicat- 
ing that this fit does provide an excellent description 
of the data (as it should) . 

3.3. Including Background Events 

In many physics analyses, there is a sample of 
non-interfering background events which can not be 
separated from the signal. We will now deal with 
this situation. The signal sample will simply be that 



of Section |3.2| thus, we will be including detector 
acceptance effects as well. For the background, we 
chose to generate it according to 3-body phase space 
weighted by a linear function in 7713^ and 

W b (9,(j>)= ^(1 + | sin0 cos 0|) (17) 

67T 

in the decay angles. The number of background 
events generated, 71&, was 10,000 (the background 
events were also subjected to the detector accep- 
tance given in (15 1). Figure]^ a) shows the ir + ir~ir° 
mass spectrum for all generated events and for just 
the background. The generated decay angular dis- 
tributions for all events, along with only the signal 
and background are shown in Figures [9jb) , (c) and 
(d). There is clearly no way to separate out the 
signal events through the use of a cut. 

3.3.1. Signal Extraction 

The method used to extract the signal events is 
described in detail in [5] . It accurately preserves all 
kinematic correlations in the data while separating 
signal and background events. Each event is assigned 
a signal weight factor, Q, or equivalently, a back- 
ground weight factor, 1 — Q. These Q-f actors are 
then used to weight each event's contribution to the 
"log likelihood" during unbinned maximum likeli- 
hood fits to extract physical observables. 

The method works in a very similar way to the 
one presented in this paper. A metric is defined in 
the space of all relevant kinematic variables and the 
n c nearest neighbor events (we chose n c — 100) are 
selected. For this example, the metric defined in ( 11 1 



can again be used. Since each subset of events oc- 
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confidence level 
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Fig. 8. (Color Online) Events generated without background and with detector acceptance given by (TH). (a) x 2 (b) pull and 
(c) confidence level distributions. The red lines represent the theoretical distributions, which contain no free parameters. 



cupics a very small region of phase space, the m 37T 
distributions can be used to determine each event's 
Q- factor while preserving the correlations present in 
the remaining kinematic variables (cos 9 and <j)) . 

To this end, unbinned maximum likelihood 
fits were carried out for each event, using its 
nearest neighbors, to determine the parameters 
a — (6qj b\, s, a) in the p.d.f. 



F(m 3ir ,d) = 



where 



B(m 37r ,a) + S(m 3n ,a) 
J (B(m 3w , a) + S(m 37r ,a)) dm 



3tt 



-,(18) 



S(m 3n ,d) = s ■ y(m 37r ,TO w ,r w , 



(19) 



parameterizes the signal as a Voigtian (con- 
volution of a Breit-Wigner and a Gaussian) 
with mass m w = 0.78256 GeV/c 2 , natural width 
r w = 0.00844 GeV/c 2 and resolution a. The pa- 
rameter s sets the overall strength of the signal. 
The background, in each small phase space region, 
was parameterized by the linear function 



B(m 37ri a) = feim 37r + b . 



(20) 



The Q-factor for the event was then calculated as 



Qi 



(21) 



where m 3w is the event's 37r mass and di are the 
estimators for the parameters obtained from the i th 
event's fit. 

Figures[9je) and (f) show the extracted signal and 
background distributions, i.e. they show the events 
weighted by Q and 1—Q respectively. The agreement 
with the generated distributions (see Figures [9jc) 
and (d)) is very good. We conclude this section by 
noting that the full covariance matrix obtained from 



each event's fit, C a , can be used to calculate the 
uncertainty in Q as 



Q 



■1N..*£ 
15 da i 



(22) 



3.3.2. Extracting Physical Observables 

The likelihood function used to obtain the spin 
density matrix elements, defined in (JsJ) , can be modi- 
fied to account for the presence of background events 
as follows: 



71 + 71;, 

-ln£ = - QilnWiOi,^). 



(23) 



Thus, the Q-factors are used to weight each event's 
contribution to the likelihood. Using the Q-factors 



obtained in Section 3.3.1 minimizing (23 1 yields 
p° m = 0.640 ± 0.009 



p\_ x = 0.051 ± 0.005 
Rep° w = 0.095 ± 0.004, 



(24a) 
(24b) 
(24c) 



which are in good agreement with the generated val- 
ues. 

3.3.3. Applying the Method 

The metric used to obtain the n c nearest neighbor 



events is again (11); however, the number of mea- 



sured events contained within each hypersphere is 
now given by 



Unit — ^ ^ Qij 
i 

with uncertainty 



1 j jk k 



(25) 



(26) 



i>fc 
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(a) (b) 
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(d) 





Fig. 9. (Color Online) (a) The mg^ distributions for all generated events (unshaded), only background events (shaded) and 
for all events weighted by the background factors, 1 — Q (dashed red), (b) <j> vs. cosd for all generated events, (c) vs. cos 8 
for only signal events, (d) <j> vs. cos0 for only background events, (e) <j> vs. cos 9 for all events weighted by signal factors, Q. 
(f) cf> vs. cos 6 for all events weighted by background factors, 1 — Q. 
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where the sums (j, k) are over the n c events used to 
calculate the i th event's Q-factor and pP is the cor- 
relation factor between events j and k. This factor 
is equal to the fraction of shared nearest neighbor 
events used in calculating the Q-factors for these 



events. Thus, (26) accounts for both the statistical 



uncertainty in the signal yield, along with the uncer- 
tainty in the signal-background separation, i.e. the 
uncertainty in the Q- factors. 

The number of predicted events, along with its un- 
certainty, is calculated the same way as in Section [3.1| 
and the z values are again obtained using ([3]). Fig- 
ure 10 shows the x 2 , pull and confidence level dis- 
tributions obtained for our simulated data set. The 
values are again in excellent agreement with the the- 
oretical distributions. This indicates that we have 
properly handled the uncertainties in the signal. The 
total x 2 /n.d.f. = 9496.75/(9714.82 - 3) = 0.978. 

Calculating the correlation factor, p> k , in (26 1 is 



very CPU intensive due to the multiple loops which 
need to be performed over the data events. In [9], 
we noted that assuming 100% correlation for all Q- 
factors provides a reasonable upper limit on the sig- 
nal yield in any kinematic region. Under this as- 
sumption, which eliminates much of the bookkeep- 
ing and CPU cycles needed to properly calculate 
fp k , the uncertainty on n m . becomes 



E<4 



(27) 



wave analysis on the data, we would also need to in- 
clude any additional kinematic variables which fac- 
tor into the partial wave amplitudes, e.g. the dis- 
tance from the edge of the 7r + 7r _ 7r° Dalitz plot (typ- 
ically included in the uj decay amplitude) . We would 
then construct the likelihood from the partial waves 
and minimize — In C using the Q- factors obtained 
by applying our signal-background separation pro- 
cedure, including the additional coordinates. 

The estimators from these event-based fits would 
then serve as input into a higher dimensional ver- 
sion of the method presented in this example. The 
procedure for obtaining z-values, however, would re- 
main almost unchanged. We would simply need to 
account for the additional relevant kinematic vari- 
ables in our metric. If the fit provides a good de- 
scription of the data, then the x 2 /n.d.f. should be 
approximately one and the squares of the standard- 
ized residuals should follow a \ 2 distribution. 

4. Conclusions 

In this paper, we have presented an ad-hoc proce- 
dure for obtaining \ 2 /n.d.f '. values from unbinned 
maximum likelihood fits which does not require bin- 
ning the data. This makes it very applicable to multi- 
dimensional problems. We have shown that these 
X 2 /n.d.f . values accurately reflect how well a given 
hypothesis describes the data. 



where the sum is again over the n c nearest neighbor 
events used to calculate Qi. 

Figure 1 1 shows the \ 2 ; P un an( i confidence level 
distributions obtained for our simulated data set un- 
der this assumption. The deviations from the theo- 
retical distributions are relatively small. The total 
X 2 /n.d.f. = 8135.96/(9714.82 - 3) = 0.838. This 
value is smaller than 1 as expected; however, it is 
still reasonably close to the value obtained when the 
errors are handled rigorously. Thus, this is a reason- 
able approximation and could be used effectively in 
situations where calculating the values of p* k is not 
possible (or, perhaps, desirable). 



3.4. Extending the Example 

To extend this example to allow for the case where 
the data is not binned in production angle, we would 
simply need to include cos 0^ M or t in the vector 
of relevant coordinates, £. To perform a full partial 
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Fig. 11. (Color Online) Events generated with background and with detector acceptance given by \15\ . (a) \ 2 C 3 ) P U U an d 
(c) confidence level distributions obtained using upper limits on the Q-factor errors (given in j27[ l). The red lines represent 
the theoretical distributions, which contain no free parameters. See text for details. 



[8] K. Schilling, P. Seyboth and G. Wolf, Nucl. Phys. B15, 
397 (1970). 

[9] M. Williams, M. Bellis and C. A. Meyer, 
|arXiv:0804.3382| (Submitted to NIM). 



11 



